Automated Distractor Generation for Fill-in-the-Blank Items Using a Prompt-Based Learning Approach

Keywords: Automated distractor generation, automated item generation, natural language processing, deep learning language models, prompt-based learning Language testing programs, like many other educational and psychological testing programs, face increasing demands for flexible test administrations. Since the COVID-19 pandemic, many language proficiency tests are offered to be taken at home with more available testing dates. [...]von Davier (2018) trained a long-short-term memory- (LSTM-) based recurrent neural network model and Hommel et al. [...]transformer-based models achieved state-of-the-art performance on a wide range of NLP benchmark tasks, such as the General Language Understanding Evaluation (GLUE;Wang et al., 2019), the Standard Question Answering Dataset (SQuAD;Rajpurkar et al., 2016), and the Situations with Adversarial Generations (SWAG;Zellers et al., 2018). A typical fine-tuning process consumes a large number of examples (oftentimes several tens of thousands), yet it is rare for a testing program to have such a large item pool. [...]we designed language prompts for distractors and leveraged the prompts in fine-tuning to address this small sample challenge.

Do you know your partner's personality through virtual collaboration or negotiation? Investigating perceptions of personality and their impacts on performance

Interpersonal skills, including collaborative problem solving (CPS) and negotiation skills, are essential in many aspects of the 21st century. With the rapid development of technologies in the past decades, it has become increasingly prevalent for collaborations, negotiations, and communications to occur virtually. Furthermore, the COVID-19 pandemic accelerated the shift from in-person interactions to virtual interactions. On the other hand, personality traits, enduring characteristics of individuals that are largely stable over time, affect a wide variety of human behaviors, including how people interact with each other. In this study, we investigated the extent to which team members' personalities, the heterogeneity in personalities among team members, and the interaction processes in virtual tasks impacted performance on these tasks with limited exposure to personal information such as appearance and voice. In addition, we examined how one perceived the team partner's personality and how people tended to project their own personality onto partners during the short-term virtual interactions. Findings suggested that higher heterogeneity in personality between partners was associated with better team negotiation performance, while it was not associated with collaboration outcomes in the CPS task. Implications of the findings and limitations of this research were also discussed. • We study how team members' personalities and the interaction processes impact performance on virtual tasks. • Relationships between personality, interaction processes, and performance differ in collaboration and negotiation contexts. • People tend to project their own personality traits onto their online partners during short-term virtual interactions. • Teams that are more heterogeneous in personality show better negotiation outcomes. [ FROM AUTHOR]

Do you know your Partner's personality through virtual collaboration or negotiation? Investigating perceptions of personality and their impacts on performance

Interpersonal skills, including collaborative problem solving (CPS) and negotiation skills, are essential in many aspects of the 21st century. With the rapid development of technologies in the past decades, it has become increasingly prevalent for collaborations, negotiations, and communications to occur virtually. Furthermore, the COVID-19 pandemic accelerated the shift from in-person interactions to virtual interactions. On the other hand, personality traits, enduring characteristics of individuals that are largely stable over time, affect a wide variety of human behaviors, including how people interact with each other. In this study, we investigated the extent to which team members' personalities, the heterogeneity in personalities among team members, and the interaction processes in virtual tasks impacted performance on these tasks with limited exposure to personal information such as appearance and voice. In addition, we examined how one perceived the team partner's personality and how people tended to project their own personality onto partners during the short-term virtual interactions. Findings suggested that higher heterogeneity in personality between partners was associated with better team negotiation performance, while it was not associated with collaboration outcomes in the CPS task. Implications of the findings and limitations of this research were also discussed.

An Investigation Into the Impact of Test Session Disruptions for At-Home Test Administrations

In response to the closures of test centers worldwide due to the COVID-19 pandemic, several testing programs offered large-scale standardized assessments to examinees remotely. However, due to the varying quality of the performance of personal devices and internet connections, more at-home examinees likely suffered “disruptions” or an interruption in the connectivity to their testing session compared to typical test-center administrations. Disruptions have the potential to adversely affect examinees and lead to fairness or validity issues. The goal of this study was to investigate the extent to which disruptions impacted performance of at-home examinees using data from a large-scale admissions test. Specifically, the study involved comparing the average test scores of the disrupted examinees with those of the non-disrupted examinees after weighting the non-disrupted examinees to resemble the disrupted examinees along baseline characteristics. The results show that disruptions had a small negative impact on test scores on average. However, there was little difference in performance between the disrupted and non-disrupted examinees after removing records of the disrupted examinees who were unable to complete the test. [ FROM AUTHOR] Copyright of Applied Psychological Measurement is the property of Sage Publications Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full . (Copyright applies to all s.)